Learning to Switch Between Sensorimotor Primitives Using Multimodal Haptic Signals

Most manipulation tasks can be decomposed into sequences of sensorimotor primitives. These primitives often end with characteristic sensory events, e.g., making or breaking contact, which indicate when the sensorimotor goal has been reached. In this manner, the robot can monitor the tactile signals to determine when to switch between primitives. In this work, we present a framework for automatically segmenting contact-based manipulation tasks into sequences of sensorimotor primitives based on multimodal haptic signals. These signals include both the robot’s end-effector position as well as the lowand high-frequency components of its tactile sensors. The resulting segmentation is used to learn to detect when the robot has reached a sensorimotor goal and it should therefore switch to the next primitive. The proposed framework was evaluated on guided peg-in-hole tasks. The experiments show that the framework can extract the subtasks of the manipulations and the sensorimotor goals can be accurately detected.